1. Field of the Invention
This invention pertains generally to image acquisition, and more particularly to automatic image focusing mechanisms based on performing multiple forms of depth estimations at each lens position to optimize blur matching for the depth estimation process.
2. Description of Related Art
Image capture devices, in particular cameras, provide an automated mechanism for obtaining a correct focus on the subject. Autofocus (AF) is typically engaged in response to applying slight pressure to the shutter-release switch, such as pressing it half-way toward shutter release. In some applications, the autofocus feature continuously maintains focus, prior to capturing the image, or while capturing an image sequence or video.
During autofocus, the camera lens is automatically focusing on a three-dimensional scene and focus is controlled with respect to depth. A number of autofocus mechanisms are known in the art. One typical autofocus mechanism searches for a peak in the autofocus curve, for example utilizing the autofocus curve from image gradients. In response to the region of analysis becoming increasingly focused, image gradients become increasingly larger. It is the object of the autofocus algorithm is to obtain the peak of the autofocus curve while requiring transitions though a minimum number of camera focus positions. However, existing focus mechanisms are slow and often fail to converge on a proper focus and may continue “hunting” for a proper focal position.
Focus is perhaps the most critical metric in capturing a desired image which has led to the development of a wide range of systems for estimating or attaining proper camera focus. As a camera-lens system has a number of related elements and characteristics, a brief discussion follows of these elements and their associated characteristics.
In general terms, the two main optical parameters of a photographic lens are maximum aperture and focal length. The focal length determines the angle of view, and the size of the image relative to that of the object (subject) for a given distance to the subject (subject-distance). The maximum aperture (f-number, or f-stop) limits the brightness of the image and the fastest shutter speed usable for a given setting (focal length/effective aperture), with a smaller number indicating that more light is provided to the focal plane which typically can be thought of as the face of the image sensor in a simple digital camera.
In one form of typical simple lens (technically a lens having a single element) a single focal length is provided and this lens is also referred to as a “prime lens”. In focusing a camera using a single focal length lens, the distance between lens and the focal plane is changed which results in altering the focal point where the photographic subject image is directed onto the focal plane. Thus, although the single focal length lens has a fixed optical relation and focal length, it is used in the camera to focus on subjects across a focal range span. The fixed focal distance of a lens should not be confused with the range of focal distances obtainable on a camera using that lens, and it will be noted that adjusting the position of the lens in relation to the focal plane alters focal distance.
In using a single focal length lens, the aperture is adjusted to select the amount of light with respect to desired shutter speed, and focus is adjusted according to subject-distance, which is also referred to as the focal distance, after which one or more images are captured. Often a macro setting is provided with a different focal length selection, on an otherwise single focal length lens, to allow capturing close-up shots. In contrast, a telephoto lens provides a very narrow angle of view with high magnification for filling the frame with an image of distant objects.
Multi-focal length lenses are usually referred to as “zoom” lenses, because image magnification can be “zoomed”, or “unzoomed” as the case may be. Zoom lenses allow the user to select the amount of magnification of the subject, or put another way, the degree to which the subject fills the frame. It is important to understand that the zoom function of these lenses, or camera-lens systems, is conceptually separate from both the focus control and the aperture control, although in practice zoom lens operation even in the best of systems still slightly impacts other settings.
It is always necessary, however, to properly focus the lens for a given subject-distance irrespective of whether a single-focal length lens or multi-focal length lens is utilized. An acceptable range of focus for a given focus setting is referred to as “depth of field” which is a measurement of depth of acceptable sharpness in the object space, or subject space. For example, with a subject distance of fifteen feet, an acceptable range of focus for a high definition camera may be on the order of inches, while optimum focus can require even more precision. It will be appreciated that depth of field increases as the focusing moves from intermediate distances out toward “infinity” (e.g., capturing images of distant mountains, clouds and so forth), which of course at that range has unlimited depth of field.
For a single focal length lens at a given aperture setting there exists a single optimum focus setting for a given distance from camera to the subject (subject-distance). Portions of the subject which are closer or farther than the focal distance of the camera will show up in the captured images subject to an amount of blur which depends on many factors that impact depth of field. However, in a multi-focal lens there is an optimum focus point for each lens magnification (lens focal length) obtainable by the lens. To increase practicality, lens makers have significantly reduced the need to refocus in response to zoom settings, however, the necessity for refocusing depends on the specific camera-lens system in use. In addition, the aperture setting can require changing in response to different levels of zoom magnification.
In early cameras the focus could only be determined and corrected in response to operator recognition and manual focus adjustments. However, in response to the critical nature of proper focus, aids were readily adopted. In more recent times, imaging devices usually provide the ability to automatically focus on the subject, a function which is generically referred to today as “auto focus”. Focus continues to be a point of intense technical development as each of the many existing auto focus mechanisms are subject to shortcomings and tradeoffs.
There are two general categories of auto focus (AF) systems which exist, (1) active auto focus and (2) passive auto focus. In active auto focus, one or more image sensors are utilized to determine distance to the focal point, or otherwise detect focus external of the image capture lens system. Active AF systems can perform rapid focusing although they will not typically provide proper focusing through windows, or in other specific applications, since sound waves and infrared light are reflected by the glass and other surfaces. In passive auto focus systems the characteristics of the viewed image are used to detect and set focus.
The majority of high-end single-lens reflex (SLR) cameras currently use through-the-lens optical AF sensors, which for example, may also be utilized as light meters. The focusing ability of these modern AF systems can often be of higher precision than that achieved manually through an ordinary viewfinder.
In one form of passive AF, phase detection is utilized, such as by dividing the incoming light through a beam splitter into pairs of images and comparing them on an AF sensor. Two optical prisms capture the light rays coming from opposite sides of the lens and divert it to the AF sensor, creating a simple rangefinder with a base identical to the diameter of the lens. Focus is determined in response to checking for similar light intensity patterns and the phase difference is calculated to determine if the object is considered in front of the focus or in back of the proper focus position.
In another type of passive AF system, contrast measurements are made within a sensor field through the lens. The system adjusts focus to maximize intensity difference between adjacent pixels which is generally indicative of correct image focus. Thus, focusing is performed until a maximum level of contrast is obtained. This form of focusing takes longer (is slower) than active AF, in particular when operating under dim light, but is a common method utilized in low end imaging devices.
Passive systems are notoriously poor at making focal decisions in low contrast conditions, notably on large single-colored surfaces (solid surface, sky, and so forth) or in low-light conditions. Passive systems are dependent on a certain degree of illumination to the subject (whether natural or otherwise), while active systems may focus correctly even in total darkness when necessary. In addition, problems arise with use of blur matching whose curves do not always fit the calibrated model.
Accordingly, a need exists for a system and method of controlling camera focusing depth estimation methods in a more robust and deterministic manner. These needs and others are met within the present invention, which overcomes the deficiencies of previously developed autofocus systems and methods.